Skip to content

net::ERR_ABORTED at "https://..." from a crawling program #9363

New issue
Jump to bottom
New issue
Jump to bottom
Closed as not planned
Closed as not planned
net::ERR_ABORTED at "https://..." from a crawling program#9363
Labels
bugneeds-feedback
@Redskinsjo

Description

@Redskinsjo
Redskinsjo
opened on Dec 6, 2022 · edited by Redskinsjo

Bug description

Steps to reproduce the problem:

  1. enclose a 'new Promise' whose body includes a 'page.goto()' then a 'page.evaluate()' inside a map that returns into a 'Promise.all()'.

const getResult = async () => await Promise.all(websites.map(site => {
const promise = new Promise(async (resolve,reject) => {
try {
await page.goto(site, {waitUntil: "networkidle2"})
const result = await page.evaluate(() => {
...
return result
})
resolve(result)
} catch (err) {
reject(err)
}
})
return promise
})

try {
console.log(await getResult());
} catch (err) {
console.log("OKOKOK", err.message);
}

// msg.text() is from a console 'on' event

Puppeteer version

^19.3.0

Node.js version

14.17.6

npm version

yarn 1.22.19

What operating system are you seeing the problem on?

macOS

Configuration file

no config file, simple node server with js scripts

Relevant log output

msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
    at HTMLDocument.<anonymous> (https://www.{site}/sites/default/files/js/js_gQZflpXLjI8oiEOVh9W7qG81ILOXaEg7B8xYt5OFDqQ.js:8924:25)
    at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
    at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340) 
msg.text() hello ok
OKOKOK net::ERR_ABORTED at https://{site}.com
msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() fancyBox already initialized
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
    at HTMLDocument.<anonymous> (https://www.{site}.com/sites/default/files/js/js_5aKlLkXJ30Pe47l8D-WslgYctBeM3RPXOkR-UpKXcJ8.js:2845:25)
    at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
    at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340) 
✨  Done in 7.00s.

Activity

Redskinsjo
added
bug
on Dec 6, 2022
Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Redskinsjo
on Dec 6, 2022
Author

seems to me that it is similar to #8246 with a newest puppeteer version

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
OrKoN
on Dec 6, 2022
Collaborator

@Redskinsjo could you please provide a minimal example that we can run? (https://stackoverflow.com/help/minimal-reproducible-example)

OrKoN
added
needs-feedback
on Dec 6, 2022
Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Redskinsjo
on Dec 6, 2022
Author

sure. stackblitz and codesandbox have issues to connect to chromium, even when I install chromium locally.
anyway, please find the reproduction here :)

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
OrKoN
on Dec 6, 2022 · edited by OrKoN
Collaborator

@Redskinsjo so you are sending concurrent navigation requests to the same page and the first navigation gets aborted by the second one. Why do you think it's a bug in Puppeteer?

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
OrKoN
on Dec 6, 2022 · edited by OrKoN
Collaborator

You'd probably want to rewrite your code code as follows:

(async () => {
  const browser = await puppeteer.launch({
    headless: false,
    executablePath: executablePath(),
  });

  const page = await browser.newPage();

  page.on("console", async (msg) => {
    console.log("msg.text()", msg.text());
  });

  const getResult = async (jobSectors) => {
    const results = [];

    for (const i of [1, 2]) {
      await page.goto("https://www.google.com/", {
        waitUntil: "networkidle2",
      });
      const result = await page.evaluate((i) => {
        return i;
      }, i);
      results.push(result);
    }

    return results;
  };

  console.log(await getResult());

  await browser.close();
})();
Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Redskinsjo
on Dec 6, 2022
Author

i had to change the URL. but I am basically crawling the website and requests are sent to different URLs. Can't Puppeteer handle multiple page.goto(url) at the same time ?

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
OrKoN
on Dec 6, 2022
Collaborator

you should create multiple pages in that case (via browser.newPage).

OrKoN
closed this as not plannedon Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    bugneeds-feedback

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @OrKoN@Redskinsjo

      Issue actions
        net::ERR_ABORTED at "https://..." from a crawling program · Issue #9363 · puppeteer/puppeteer